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Abstract 

Background: High maternal mortality in India is a serious public health challenge. Demand side financing 
interventions have emerged as a strategy to promote access to emergency obstetric care. Two such state run 
programs, Janani Suraksha Yojana (JSY)and Chiranjeevi Yojana (CY), were designed and implemented to reduce 
financial access barriers that preclude women from obtaining emergency obstetric care. JSY, a conditional cash 
transfer, awards money directly to a woman who delivers in a public health facility. This will be studied in Madhya 
Pradesh province. CY, a voucher based program, empanels private obstetricians in Gujarat province, who are 
reimbursed by the government to perform deliveries of socioeconomically disadvantaged women. The programs 
have been in operation for the last seven years. 

Methods/designs: The study outlined in this protocol will assess and compare the influence of the two programs 
on various aspects of maternal health care including trends in program uptake, institutional delivery rates, maternal 
and neonatal outcomes, quality of care, experiences of service providers and users, and cost effectiveness. The 
study will collect primary data using a combination of qualitative and quantitative methods, including facility level 
questionnaires, observations, a population based survey, in-depth interviews, and focus group discussions. Primary 
data will be collected in three districts of each province. The research will take place at three levels: the state health 
departments, obstetric facilities in the districts and among recently delivered mothers in the community. 

Discussion: The protocol is a comprehensive assessment of the performance and impact of the programs and an 
economic analysis. It will fill existing evidence gaps in the scientific literature including access and quality to 
services, utilization, coverage and impact. The implementation of the protocol will also generate evidence to 
facilitate decision making among policy makers and program managers who currently work with or are planning 
similar programs in different contexts. 
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Background in 1990, and was 251 per 100 000 live births in 2008 [2]. 

The adoption of the Millennium Development Goals in Despite the international commitment, maternal mortal- 
2001, specifically goal 5, renewed the global emphasis on ity remains somewhat unchanged globally. In 2008, an 
reducing maternal mortality. This international commit- estimated 342,900 women died world-wide from preg- 
ment gained political attention; many low income coun- nancy related complications. More than half of these 
tries prioritized improving women's access to maternal deaths occur in six low-income countries; one-quarter in 
health services [lj.The global MMR decreased from 320 India alone [2]. 

Between 1992 and 2006, the Indian Ministry of Health 
and Family Welfare focused on strengthening the health 
system infrastructure to better support emergency ob- 
stetric care (EmOC) services especially in the public 
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sector. A concerted effort was made to increase capacity 
for institutional deliveries by upgrading facilities to im- 
prove access to skilled birth attendance and EmOC [3]. 
Key strategies included upgrading community health 
centers to function as first referral units, creating 24/7 
access to health centers for delivery, and training and 
empowering non-specialist qualified medical doctors to 
administer anesthesia for emergency obstetric proce- 
dures [1]. In spite of this government investment, na- 
tional surveys showed that the proportion of 
institutional deliveries only increased marginally from 
26% to 39% during that period [4,5]. 

While it is extremely important to invest in EmOC fa- 
cilities, developing strategies that increase the use of 
these services especially among the poor who suffer the 
largest burden of maternal deaths are equally significant 
[6]. Most deaths can be avoided by prompt access to 
EmOC services. However, despite strengthening of the 
supply side (facilities), poor women are still at risk as 
they face a number of barriers, particularly financial, to 
access EmOC in the absence of social safety nets and 
widely prevalent out-of-pocket payment mechanisms [7]. 
In 2005, the government recognized the need to 
prioritize maternal health amongst the poorest and set 
up the National Rural Health Mission (NRHM) specific- 
ally to improve access to care for low income people in 
rural areas. 

Using the platform of the NRHM, the government 
introduced demand-side financing initiatives for maternal 
health to reduce financial barriers that often preclude 
women from accessing skilled attendance at birth [8] . Two 
large scale demand-side financing programs to promote 



institutional delivery, conditional cash transfers (CCT) 
and vouchers, are currently in operation. The intended 
outcome of both programs is to promote institutional de- 
livery, thus reducing maternal mortality through improv- 
ing access to EmOC. 

A traditional CCT provides a monetary incentive dir- 
ectly to the intended group on the condition the benefi- 
ciary satisfies a pre-defined set of requirements [9]. 
India's Janani Suraksha Yojana (JSY or Safe motherhood 
program), a nationwide program, is a conditional cash 
transfer to all women who deliver in public sector insti- 
tutions. This program will be studied in Madhya Pradesh 
Province. Chiranjeevi Yojana (CY or Long life program) 
is operational only in the Gujarat province, and func- 
tions in the context of an existing strong private obstet- 
ric care sector. It is a voucher based system where 
private obstetricians are reimbursed by the government 
to perform deliveries of tribal women or those living 
below the poverty line. Vouchers, which tend to be cash- 
less, are used to reduce the direct costs of healthcare 
and increase demand for services [10] (Table 1). 

This protocol describes the proposed comprehensive 
evaluation of two respective programs in Gujarat and 
Madhya Pradesh. The evaluation, called the MATIND 
project is a four year project supported under the EU 
FP7 framework. The proposal aims to study (i) trends in 
program uptake since inception(ii)the influence of the 
program on institutional delivery rates, maternal and 
neonatal outcomes, (iii) explore service providers' 
experiences, (iv) emergency transportation systems for 
obstetric referrals, (v) characteristics of the facilities par- 
ticipating in the programs, (vi) quality of care 



Table 1 Characteristics of two Indian demand-side financing programs for maternal health 




CY - a voucher program (Gujarat) 


JSY - a conditional cash transfer (MP) 


Area of Operation 


Gujarat (60.4 million) 


Nationwide (studied here in Madhya Pradesh) 
(72.6 million) 


Context 


Socioeconomically relatively advanced: MMR half of that in 
MP (142/100 000), 16% population below the poverty line 


Poor socioeconomic indicators, MMR (310/100 000), 
largely rural province, 38% of the population 
below the poverty line 


Target group 


Mothers below poverty line and tribal mothers 


All mothers 


Type of Program 


Payment by state to the private provider i.e. 

obstetrician (voucher based) 


CCT - payment by state to the mother 

(conditional cash transfer) 


Incentive for 


Institutional delivery 


Institutional delivery 


Accepted place of delivery 


Empanelled private sector 
obstetric facilities 


Public sector institutions largely 


Involvement of private sector 


Yes 


Yes (extremely restricted) 


Payment mechanism 


Providers paid per block of 100 deliveries (20% in advance) 


Payment to woman at the time of discharge from 
hospital after delivery 


Quantum of payment 


A flat amount of INR 288000($5760) is paid to the obstetrician 
per 100 women (regardless of delivery type) 


INR1400 ($28) to rural mothers on discharge 
(INR1000 to urban mothers). 


Expected effect 


Provides access to EmOC care - which is available more widely 


Increase institutional delivery, hence access to EmOC 


Support for Emergency 
transport 


Centralized 108 ambulance system 


Decentralized Janani Express model 
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administered, (vii) experiences of the beneficiaries and 
non-beneficiaries, (viii) the role of community health 
workers and (ix) the cost effectiveness of the programs. 

The program evaluation will be conducted at three dif- 
ferent levels of the health system with specific objectives 
at each level: (i) the provincial (policy) level which 
includes secondary data from governments and other 
important stakeholders at the provincial level, (ii) the 
district level (sub-provincial) which focuses on health fa- 
cilities (iii) and the community level, in villages (sub-dis- 
trict), which includes mothers and local health workers. 

The objectives at each level of the study are listed 
below and summarized in Table 2: 

1. Provincial (policy) level: 

(i) Program Trends: To describe trends in program 
uptake for each program since inception and to 
study district level time series of program uptake 
data from 2004 to 2011. Further to examine the 
extent to which changes in the proportion of 
institutional deliveries have influenced maternal 
and neonatal outcomes over that period of time. 

(ii) Private Sector Participation: To describe trends in 
private sector (health facilities) participation in each 
of the programs over time. Also, to study the 
characteristics of private sector participant facilities. 
With regard to the JSY program in MP, to 
explore the reasons for the decision to 
subsequently reduce the role of the private sector 
in the program. Also to explore among selected 
erstwhile participating private facilities, their 
experience with JSY and their perception on the 
state decision to exclude them from the scheme. 
With regard to the CY program, to explore 
experiences and motivations among selected 
private providers currently participating in the 
program, reasons for non-participation among a 
sample of those who were eligible but never 
participated and those who participated initially 
but have since left the program. 

(iii) Emergency transportation for obstetric care: To 
study two different emergency obstetric 
transportation models utilized in each of the 
programs with regards to coverage, equity in 
access, current utilization, and cost of the service. 
Further, to explore among a group of mothers 
their experience of gaining access to and using 
the transportation. 

(iv) Cost-effectiveness of the Programs: To develop 
and apply a model for assessment of the utility of 
the two maternal health programs and to 
perform a sensitivity analysis by testing 
alternative assumptions for a cost utility analysis. 



2. Facility Level (public and private facilities): 

(v) Facility Survey: To list all facilities in the study 
districts offering obstetric care and classify them 
as non-EmOC, basic or comprehensive 
emergency obstetric care (BEmOC or CEmOC) 
facilities. Further, to study the characteristics of 
these facilities with regard to ownership, 
accessibility, bed strength, human resources, 
training, referral communications and processes, 
availability of equipment, drugs and supplies and 
infection control procedures. 

(vi) Quality of Care: To study in EmOC facilities the 
quality of care through direct observation of 
normal deliveries, a review of case records and 
registers and assessments of provider competence. 
Further to study experiences and perceptions of 
quality of care of mothers who delivered under the 
respective programs, and to study key procedures 
they experienced. In cases of mothers who were 
referred into facilities, referral indications and 
transportation details will be studied. 

3. Community level (mothers in sampled study 
villages): 

(vii) Program beneficiaries and non-beneficiaries: To 
study and compare beneficiaries and eligible non- 
beneficiaries of the program in the community in 
terms of: socio-demographic characteristics, 
utilization of antenatal services, birth attendance, 
delivery outcome, birth weight, receipt of post natal 
care, immunization of the child and costs of care 
incurred. Reasons for participation/non 
participation and choice of place of delivery will be 
explored. Predictors of non- participation will also 
be studied. 

(viii) Equity in access to facilities (geographic): To 
estimate the influence of distance to facilities on 
place of delivery in comparison to other 
determinants of delivery service use. 

(ix) Role of community health workers in program 
uptake: To study the role played by the ASHA 
(accredited social health activist) and the 
traditional birth attendants (TBA) in the 
antenatal, delivery and postnatal periods of 
program mothers. Also to explore among ASHAs 
their motives for participation and long term 
professional aspirations within the NRHM. The 
dynamic between various community level 
workers, the TBA, the ASHA and the female 
health worker will be explored. Concerns on the 
potential duplication of roles between the latter 
two will be explored among policy makers. 
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Table 2 Summary of objectives and proposed methods at the implementation level 



Study level Objectives 



Proposed methods 



Provincia 
(Policy) 



District 
(Facility) 



Community 



To study program utilization trends 

To ascertain the influence of changes in ANC service 
utilization and institutional deliveries on maternal and 
neonatal outcomes 

To study trends in private sector participation 

Explore the motivations for participation/non 
participation with the private sector in CY program 

To understand the exclusion of the private sector in the 
JSY program 

Explore experience of private provider in JSY 

Comparison of emergency care transportation systems 

Cost effectiveness of the programs and the 
emergency transportation models 

List and classify all facilities performing deliveries 
according to their EmOC functionality 

Survey and describe facilities conducting deliveries 

Assess the quality of care administered in study facilities 

Study outcomes (type of delivery, maternal 

mortality and morbidity, foetal outcomes) among program 

beneficiaries and non-beneficiaries in study facilities 

Experiences of public and private sector physicians 
of the programs (in terms of task load, shifting of tasks, 
human resources required, monetary transactions 
with the program) 

Identify program beneficiaries and 
non-beneficiaries. 

Study differences between background characteristics, 
geographic access to EmOC, outcomes (maternal), type 
of health service utilization in pregnancy (and delivery), 
use of emergency obstetric transportation services 
and expenditures during delivery 

Identify predictors for program uptake 

Compare infant health status and health 
service utilization between users and non-users. 

Among users, study perceived quality of care at facility 

Explore motives and barriers for participation/non-participation 
among eligible beneficiaries and non-beneficiaries 

Understand the working dynamic for the 
community health workers(ASHA, TBA & 
female health workersjand their role in the program 



Secondary data analysis - complete times series, Document reviews, 
Key person interviews with policy makers, program managers and 
private sector representatives, Stakeholder discussions 



Facility Survey, Observations of normal deliveries, Register and 
case record review, Case vignettes to study provider competence, 
In depth exit interviews with mothers 



Questionnaires, In-depth interviews, Focus group discussion, 
Spatial methods (GIS) to study distance 



Methods 

Setting 

The study will take place in two provinces (Figure 1) 
where the demand- side financing programs are oper- 
ational, Madhya Pradesh and Gujarat. These provinces dif- 
fer from one another both socio economically and with 
regard to other health related indicators (Table 3). 

For studies at the district level, we purposefully selected 
three districts from each province in collaboration with 
the provincial ministries of health. The selected districts 
include districts from different geographic areas of each 



province, varying human development indices and repre- 
sent different population sub groups including disadvan- 
taged groups like scheduled castes (SC) and tribes (ST). 

Study design 

A range of methods will be utilized during the data col- 
lection phase. Document reviews, time series, surveys, 
in-depth interviews, exit interviews, focus group discus- 
sions, case record reviews, and structured observations 
will be used. These are described in greater detail in the 
section below. 
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MATIND STUDY AREA 
UJARAT AND MADHYA PRADESH 
STATE - DISTRICT 



VTTAR PRADESH 




............ , fManoi 




INTERNATIONAL BOUNDARY 
STATE BOUNDARY 
COASTAL BOUNDARY 
PILOT DISTRICT AREA 
STUDY DISTRICT AREA 
DISTRICT BOUNDARY 



Figure 1 MATIND Study Area: Selected Study Districts in Madhya Pradesh and Gujarat. 



Data collection methods 

1. Provincial (policy) level: 
At this level we will collect secondary data on 
program related inputs, outputs and outcomes in 
each province. A time series analysis, consultations 
with key stakeholders, and review of relevant 
program documents will allow us to study program 
trends (objective i) and comment on the uptake of 
the programs since inception. A complete time series 
will be derived from maternal health data with regard 
to uptake (proportion of eligible women who 
participate), institutional delivery (total number of 
women delivering in a public or private facility), 
district level ante-natal care (ANC) coverage 
(proportion of women completing >3 ANC visits), 
and maternal and neonatal deaths. The data will be 
obtained from multiple sources including the District 
Level Household Survey (DLHS), Annual Health 
Survey, Sample Registration System (SRS), Vital 
Registration and health department sources for the 



purpose of data triangulation and the development of 
robust estimates. A major challenge in the analysis is 
the lack of complete time series for the maternal 
mortality data. Instead of relying on a single source 
for information, statistical data fusion techniques will 
be applied to collate data from various sources. The 
data fusion techniques will borrow strengths across 
time and space from the existing data to derive a 
complete time series estimates for the variables of 
interest. 

A second study at the policy level will focus on 
trends in private sector participation (objective ii) in 
each of the programs over time. The role of private 
providers varies in each program. Private providers 
played a small part in the JSY program; however they 
are a large resource in Gujarat and thus the central 
focus in CY. Private facility characteristics will be 
studied through a cross-sectional facility review (see 
objective v). In MP, we will explore with NRHM 
program managers the experiences of collaborating 
with private providers and the perception of the role 
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Table 3 Demographic and maternal health indicators of Gujarat, Madhya Pradesh 





Population 


Administrative districts 


BPL 


Literacy rate 


Institutional delivery 


IMR 


MMR 


Madhya Pradesh 


72.6[25] 


50 


38%[26] 


71%[25] 


47%[27] 


62[28] 


310[29] 


Gujarat 


60.400] 


25 


16%[31] 


79%[32] 


56%[33] 


44[28] 


142[28] 



of the private sector in JSY, justification for criteria 
used for selection, and rationale for their inclusion in 
and subsequent exclusion from the program through 
in-depth interviews. Implementation issues and the 
overall benefits or disadvantages of participating in 
the program will also be explored with a selected 
group of private providers. 

In Gujarat, we will use qualitative methods to explore 
participation of the for-profit health sector in CY and 
understand the factors influencing private provider 
participation. The qualitative research will be 
conducted using an interpretive approach, and will 
seek to understand private providers motives for 
participation/non participation, appropriateness of 
payment amounts under the program, procedures for 
reimbursement and interfacing with government 
officials. In addition, the study will explore how the 
program affects/affected patient turnover, what 
proportion of their patients are/were CY, influence 
on human resources, changes made to the facility to 
incorporate the program and suggestions on how to 
improve the functionality of the program. An 
experienced researcher will interview a purposive 
sample of private providers who (i) are currently 
participating in the program, (ii) were part of the 
program but no longer participate and (iii) never 
participated. Thematic analysis using the framework 
approach will be applied. 
Difficulties accessing transportation to receive 
emergency obstetric care is one of the numerous 
barriers experienced by many rural women [11], and 
we plan to explore the emergency transportation 
alternatives for obstetric care (objective iii) in each of 
the programs using quantitative and qualitative 
methods. The two programs use different emergency 
transportation models both of which are public 
private partnerships initiated by their respective 
governments. The '108' emergency transport scheme 
in Gujarat was launched in 2004. '108' is a 
centralized, free, fully integrated emergency service 
ambulance that is equipped with trained staff and 
equipment to administer lifesaving care en route. It is 
a common transportation means used for all medical 
emergencies and disaster situations. Janani Express 
Yojana (JEY or Maternal Express Program), piloted 
in 2006 and scaled up in 2008, is utilized in Madhya 
Pradesh. It is also a free ambulance service; however 



it is used exclusively to transport obstetric cases to 
public facilities. It is a more basic form of 
transportation, and does not provide amenities to 
treat patients en route to the hospital. JEY is a 
decentralized service; vehicles are hired on a 
contractual basis locally at the district level and 
utilize a decentralized call center. 
A quantitative retrospective data analysis will be 
completed with data ascertained from call centers to 
determine utilization patterns and uptake of the 
respective transport systems. Socio-demographic 
characteristics of the beneficiaries and other 
outcomes will be collected in a cross-sectional 
questionnaire to study equity in use of the service. 
NRHM program officials and District Chief Medical 
& Health Officers (CMHOs) will be interviewed to 
gain insight into the structure under each model and 
the benefits and limitations of the respective 
decentralized and centralized emergency 
transportation services. The discussion will include 
perceptions on efficiency, geographical accessibility, 
functionality, sustainability, technical capacity, 
constraints and motivations of the human resources, 
and procedural limitations. In addition, costs 
associated with implementation and ongoing usage 
will be elicited. 

All interviews will be conducted in Hindi by a 
research assistant, transcribed and then translated 
into English. In-depth interview data will be analyzed 
thematically, using the framework approach. The 
analysis will draw out differences between the two 
schemes in terms of implementation, stakeholder 
views of the schemes and experiences of utilization. 
A cost effectiveness analysis between the two 
programs is also planned. 

A cost effectiveness of the two programs (objective 
iv) will be performed through two different methods. 
The first approach will be to compare the costs and 
outcomes of the two programs for matched groups 
of participants. The primary outcome will be the 
proportion of institutional delivery. Other outcomes 
to be analyzed will be the proportion of cesarean 
deliveries, antenatal check-ups and delivery outcomes 
(live births, stillbirths, miscarriages). The costs will 
primarily be from the payer perspective, the state, 
but also costs to the patients will be collected and 
analyzed. The second approach will be to analyze 
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each system separately from an equity perspective, i.e. 
how costs and access to services are distributed 
according to categories of socio-economic status. In 
view of the complexity and multi-dimensionality of 
both the interventions and the outcomes, we will not 
aim to comprise the analysis in a single index number. 
2. Facility Level: 
At the facility level we will collect primary data on 
the types of facilities that are administering obstetric 
care under the programs (objective v), and the 
quality of care received (objective vi). These facilities 
will be visited to study maternal health outcomes. 
Both quantitative and qualitative methods will be 
utilized to achieve these objectives. 
An initial list of all public and private facilities in the 
study districts will be obtained from government 
records. The list will be verified by site visits. 
Snowballing will then be utilized to identify any 
missing private facilities where obstetric care is 
provided. A modified Averting Maternal Death and 
Disability (AMDD) survey [12] will be administered 
to each facility by a member of the research team 
and the performance of EmOC functions will be 
ascertained by interviewing key staff. The facilities 
will be classified with reference to standards based 
on the UN and WHO definitions [13] as non-EmOC, 
Basic Emergency Obstetric Care (BEmOC) or 
Comprehensive Emergency Obstetric Care 
(CEmOC). The availability level of EmOC services 
will be ascertained with reference to the population; 
the number of CEmOC, BEmOC and non-EmOC 
facilities in the public and private sector and in the 
rural and urban areas will be calculated. 
A cross-sectional survey will provide additional 
information on how the facilities function under the 
respective programs as well as aspects of quality of 
care provided. The assessment will include variables 
on infrastructure, accessibility, human resources, 
functional equipment, accessibility to drugs and 
supplies, referral communications and processes, 
infection control procedures and service statistics. 
This survey will provide the current operating state 
of the facilities. Descriptive statistics will be 
calculated for the areas mentioned above. These will 
be compared between public and private facilities 
and between the two programs. 
Data on delivery outcomes for women admitted in 
the facility will also be collected over a five day 
period at each facility that had more than 10 
deliveries in a month. A research assistant, assigned 
to the facility, will administer a short questionnaire 
to mothers who deliver within the defined time 
frame. Basic socio-demographic characteristics, 
pregnancy and delivery details, and indication for 



referral if appropriate will be elicited. More specific 
details of the same delivery will be obtained from the 
nurse in the labor ward. These will include type of 
delivery, presence of known pregnancy or delivery 
related complications, maternal and neonatal 
outcomes. This will provide information on the 
proportion of these events among program mothers. 
In the case of CY, it will allow a comparison with 
non-program mothers who deliver at the same time 
in the program hospitals. This study will provide a 
cross sectional snap shot of outcomes for program 
mothers. The proportion of program mothers who 
have complications, who undergo instrumental 
delivery, referred, who have adverse neonatal 
outcomes will be reported as will the number of 
maternal deaths during this period. 
The Hulton et al. Quality of Care framework will be 
used to assess the quality of care (objective vi) from 
both the provision of care (facility perspective) and 
the experience of care (user perspective) [14]. 
Observations of normal deliveries, case record and 
register reviews, competency rating of service care 
provider, mother's experience and the referral 
process will all be used to ascertain the quality of 
care being administered to obstetric patients. 
Non-participant observation of normal deliveries will 
be performed on a sample of mothers in selected 
facilities. The observations will describe the technical 
care given during labor and delivery, infection 
control practices and the interaction between the 
staff and mother. A retrospective review of pre- 
defined indicators from a sample of case records will 
be performed in all study facilities to assess the 
quality of records maintained. The registers will also 
be assessed and the number of deliveries and 
procedures performed will be recorded. The 
competence of labor ward nurses providing obstetric 
care will be assessed utilizing case vignettes. The case 
vignettes will focus on normal labor and delivery. A 
short structured exit interview will be administered 
to all post-partum mothers enrolled in the study 
before discharge to determine which key procedures 
they underwent. The procedures included in the 
questionnaire will be based on common practices 
that may harm or benefit the mother during or after 
delivery, as described in the WHO evidence based 
practices [14,15]. A sub-set of women who 
participate in the exit interview will be selected for 
an in-depth interview that will take place in their 
homes within four weeks of delivery. The in-depth 
interview will explore her experience of delivery care 
under the program and explore potential aspects of 
care that may inhibit her utilization of the facility for 
a future delivery. Indications for referral and referral 
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processes for women referred to facilities at different 
levels will also be studied. Mothers who have been 
referred will be interviewed to obtain details of 
indications for referral in, distance travelled, time 
taken, transport and costs. 

For the observations, content analysis will be used to 
conceptualize and classify events, behaviors and 
actions to obtain a rich understanding of quality of 
care and explanation of how women experience care 
at facilities [16]. Basic descriptive statistics will be 
calculated for key indicators from the record and 
register review. With regard to provider competence, 
descriptive analysis will identify competence areas 
that require strengthening. The data will be analyzed 
using inferential statistics to compare scores between 
levels of facilities and qualification and experience of 
providers. The data from exit interviews will be 
analyzed to describe use of evidence based practices 
by level of facility. Qualitative data derived from the 
in-depth interviews with mothers will be analyzed 
using the thematic Framework approach [17]. The 
obstetric referrals, indications for referral and details 
of referral-transport, costs, distance, and time will be 
described by the level of facility using mean, standard 
deviation and tests of significance. The referral 
pathways of the maternal deaths will be mapped on 
GIS and buffer analyzed. The buffer analysis will be 
used to study referring facility locations with regard 
to transfer time to CEmOC facilities. 
3. Community level (mothers in the study villages): 
At the community level we will collect primary data 
from recently delivered mothers to compare 
outcomes between eligible beneficiaries and non- 
beneficiaries (objective vii) for each of the programs. 
A cross-sectional study will be conducted with 
women residing in the selected study villages who 
delivered in the last year. JSY program participation 
is defined as an institutional delivery in any public 
health facility. Program non participation is defined 
as a home or private facility delivery. CY program 
participation is defined as a below poverty line (BPL) 
or tribal mother who delivers in a CY approved 
facility. Program non participation is defined as BPL 
or tribal mother who delivers outside of a CY 
approved facility. 

Stratified systematic random sample will be used to 
select the study villages in each district. The districts 
will first be stratified by blocks (administrative units) 
and then the villages in each block will be classified 
into two strata based on the size of the population. 
Villages will be randomly selected from each strata 
based on the population proportion. Given the 
different design of each program, the sample sizes 
will vary in each province. 



There will be three points of data collection; (i) when 
the woman delivers, (ii) 15 days and (iii) 28 days 
post-delivery. The ASHA will administer a simple 
symptom based checklist to all mothers regardless of 
place of delivery to ascertain maternal morbidity, 
complications associated with the pregnancy and 
delivery, at the three time points. To ensure quality 
data collection, a physician will administer a second 
checklist to 5% of all mothers which will be 
compared to the data collected by the ASHA. 
Socio-demographic details, history of previous 
pregnancies, receipt of comprehensive maternal 
health care (antenatal, intrapartum and post natal 
care), place of delivery, delivery type, birth weight, 
immunization of the child, indirect and direct costs 
associated with delivery, program participation and 
maternal and neonatal outcomes will be elicited post- 
delivery in the mother's home by a trained research 
assistant. In addition, reasons for participation/non- 
participation in the program, reasons for their choice 
of facility and barriers (if any) to access of care will 
be collected. Data will also be collected on household 
amenities and assets. This will help create an 
indicator to assist in quantifying the degree of 
income related inequalities and how it influences the 
inequities in access to care. In the event of a 
maternal death, the head of the household will be 
requested to provide the information. 
Propensity score matching will be used to select a 
comparison group from the sample of non- 
participants closest in terms of observable 
characteristics to a sample of program participants. 
Comparisons of the outcome variables described 
above between the two groups (users and non-users) 
will be studied. Reasons for participation/non 
participation will be described. In a separate 
hierarchical model, the predictors of non- 
participation will be studied in each program. Costs 
incurred will be compared by place of delivery. 
The facilities identified in objective v will be mapped 
onto a Geographic Information System (GIS) 
(objective viii). Digitized GIS maps of the selected 
districts in both provinces will be created. The map 
will have basic layers of district and block 
boundaries, road and rail networks, and settlements. 
Global Positioning System (GPS) co-ordinates will be 
taken for all facilities and added to the map as a 
separate layer. The location of the mothers who 
participated or did not participate in the program 
(objective vii) will be added as an additional layer. 
Distance from the mothers' village to the selected 
health center will be calculated from the GIS. The 
association between distance from a facility and 
program use will be studied. Level of care offered at 
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the facility will also be used as a predictor in the 
model for choice of place of delivery. Suitable GIS 
software along with standard statistical software will 
be used. A regression model will be developed using 
the background predictors of place of delivery 
(maternal characteristics, socioeconomic situation) as 
well as distance to the nearest facility. 
The last objective in this level will focus on the 
community health workers' participation in the 
program (objective ix). The ASH A is a female 
resident of the village who is incentivized by the 
government to motivate women to deliver at facilities 
under the programs. The government has identified 
the ASHA as a pivotal role in the community success 
of the program [18]. A purposive sample of ASHAs 
and mothers from their villages will be identified and 
invited to participate in separate focus group 
discussions and individual in-depth interviews. The 
following areas will be explored within the two 
groups: their practical role in the program, 
experience recruiting and retaining mothers in the 
program (cooperation received and barriers 
encountered), influence on the decision making 
process of the mothers on where to deliver, types of 
services provided to the mother, the interaction with 
other ground level health workers, perception of how 
their activism is received within the community 
(voluntary, enforced or incentivized activism), and 
long term career aspirations. In addition, a sample of 
TBAs will be identified and interviewed about their 
role in the program and how they interact with the 
mothers, ASHAs and other community level health 
workers. Among NRHM program officials at state 
level, the risk and implications of overlapping roles in 
the community will be explored through in-depth 
interviews. Data from all interviews and group 
discussions will be analyzed using thematic 
Framework analysis [17]. 

Data management 
Quantitative studies 

Research Electronic Data Capture (RED Cap) will be uti- 
lized as an online database. Data will be exported into 
appropriate statistical software for analysis. Qualitative 
studies: Transcripts and recorded material will be stored 
in a locked location. The interviewee will be disasso- 
ciated from the transcripts before analysis. 

Discussion 

Both of the proposed programs are innovative demand 
side financing programs with an explicit focus on promot- 
ing institutional delivery. They are the first of their kind 
implemented on such a large scale. Several demand-side 
financing interventions have been previously deployed to 



create demand for different health services [19]. Most of 
these schemes focus on vulnerable groups and have pro- 
vided incentives for child education, immunization and 
nutrition. Demand side programs have varied from being 
small scale pilot programs to larger nationwide programs 
like the PROGRESA/Opportunities program in Mexico, 
the BolsaFamilia in Brazil and others in Latin America 
[20]. To date, there have been some government run na- 
tional or sub national level demand side programs focused 
on maternal health, particularly in South Asia [21]. 

In this article we have outlined the protocol for an 
evaluation of the two large scale demand side financing 
programs for maternal health in India. There have been 
repeated calls in the scientific literature to study the effect- 
iveness of such large demand side financing programs. 

Design of the evaluation: Habicht et al [22] emphasizes 
the key influence of 'why an evaluation is done' on the 
subsequent design of the evaluation. The main reason 
for the proposed evaluation is to generate evidence on 
the performance and impact of these programs that is 
relevant to policy makers and program managers in 
India and other countries, who currently work with 
these programs or are considering initiating comparable 
programs in similar or different circumstances. From a 
country perspective, the evaluation will provide feedback 
on program performance and impact in the study pro- 
vinces. It will also suggest areas that need to be strength- 
ened. The evaluation aims to bridge evidence gaps in the 
scientific literature on the performance and impact of 
large scale demand side financing programs for maternal 
health. 

The current evaluation studies both performance and 
impact [22] of the programs. This is a comprehensive de- 
sign covering provision including access and quality to 
services, utilization, coverage and impact in comparison 
to reports on these programs thus far. A recent impact 
evaluation of the CCT program in India [23] based on sec- 
ondary data reported a significant increase in institutional 
delivery. Previous literatures on the effects of CCT have 
demonstrated while the use of services show improve- 
ment, there is mixed evidence on the final outcome [20]. 
This evaluation also aims to study the influence of the 
programs on the maternal mortality outcome, which has 
not been studied before. 

Multiple methodologies will be used in the evaluation 
to study the performance of the program. These will in- 
clude quantitative analysis of secondary data, surveys of 
facilities, questionnaires to mothers, community based 
surveys of households, and spatial analyses to study ac- 
cess. Qualitative methods will also be employed. These 
will include in depth interviews with policy makers, pro- 
gram managers, healthcare providers and women. Focus 
groups with women who utilized the program will also 
be done. The project will contribute new knowledge in a 
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number of areas which have not been studied before in 
the context of demand side financing programs; includ- 
ing spatial analysis to study access to EmOC, private sec- 
tor collaboration in demand side programs, an 
assessment of potential models for emergency transport 
and quality of care under the programs. 

The assessment of impact will be made from secondary 
and primary data. One measure of impact will be the in- 
fluence of the programs on the final outcome i.e. maternal 
mortality. This will involve the construction of statistical 
estimates from secondary data sources as detailed on page 
9. A second assessment of impact will be to study differ- 
ences in outcomes among eligible beneficiaries and non- 
beneficiaries. While a randomized experimental study 
would have been the most robust design (balancing con- 
founders between users and non-users), this is not feasible 
for these specific large programs as they have been rolled 
out nationwide/statewide. Hence, there are no non- 
program groups from which a random sample can be 
drawn. Given that the two programs are implemented 
completely across the provinces under study and so pre- 
clude a randomized design for evaluation, we will generate 
the comparison groups (counterfactual) using matched 
comparisons. In this evaluation, propensity score match- 
ing [24] will be used to select a comparison group from 
the sample of non-users closest in terms of observable 
characteristics to a sample of program participants. 

In addition to performance and impact of the two pro- 
grams, a cost utility analysis of each of the programs from a 
payer (state) and patient perspective will be done. There have 
been no previous economic analyses of these programs. 

In summary, this paper describes a comprehensive evalu- 
ation protocol for two innovative demand side financing 
programs to promote maternal health in India. The proto- 
col comprises an assessment of the performance and im- 
pact of the programs as well an economic analysis. The 
implementation of the protocol will generate evidence to 
facilitate decision making among policy makers and pro- 
gram managers who currently work with or are planning 
similar programs in different contexts. The evaluation also 
aims to fill existing evidence gaps in the scientific literature 
on the performance and impact of large scale demand side 
financing programs for maternal health. The results from 
the studies in this protocol will help support the design of 
efficient demand side financing programs in the future. 
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